In today's connected world, enterprises expect their employees to access external websites as a means for boosting productivity by leveraging what the Internet today offers in terms of enhanced collaboration, communication and information gathering and sharing. However, this also opens up these enterprises to potential malicious attacks when employees access websites that can be source of malware or phishing attacks, which, in turn, expose the entire internal enterprise network to malicious attacks. In addition, there have been several recent cases of employees (either intentionally or inadvertently) leaking enterprise trade secrets or confidential data as a result of unfettered access to the cloud.
Because of this, enterprises today recognize the need for URL (uniform resource locator) filtering as applied to their employee needs to access the Internet. Specifically, enterprises have shifted away from providing unfettered access to the Internet to defining strict policies on what URLs or URL categories employees can access, in order to avoid leaks of sensitive information (e.g., company assets or customer data), legal issues due to employees accessing unauthorized content, and loss of productivity due to employees accessing social networking sites.
During the infancy of the Internet, URL filtering was merely implemented via firewall rules but as the Internet quickly grew from a few thousand to millions of URLs, that strategy quickly became unusable due to issues with scale. URL filtering quickly evolved into deploying an appliance inline with all network traffic thereby enabling visibility into all outgoing network access. As the load on these appliances grew, specialized appliances that only inspected outgoing web traffic became the norm. These appliances were sometimes deployed as explicit proxies or in other cases as transparent proxies, also known as Secure Web Gateways (SWGs).
These proxies look at the header of every URL headed to the Internet, identify the URL being accessed and lookup the URL being accessed in a database that categorizes the URL into different Web categories, such as entertainment, social networking, news, malware, etc. The categorized URL is then compared against the policy defined by the enterprise to come up with an enforcement decision on the URL. Access to the URL is then allowed or denied.
The above approach falls short when HTTPS is used to access external URLs. Since the headers are already encrypted by the time the outgoing URL request reaches the proxy, the proxy cannot categorize the URL. Enterprises solve this by running SSL proxies that have to decrypt the outgoing traffic in order to categorize the URL and re-encrypt to preserve the integrity of the secure connection.
There are several drawbacks with the above approaches. In case of unencrypted traffic, the proxy has to be explicitly configured on each client or all traffic destined to a particular port (e.g., port 80) has to be redirected to the proxy via a Layer 4 switch configuration or other redirecting mechanisms (in case of transparent proxies). For transparent proxies, redirecting traffic based on port number is prone to being defeated by web servers not running on standard ports. In either case, it is difficult to create URL policies based on user identity.
In the case of encrypted traffic, in addition to the above drawbacks, performance becomes a major issue considering that the SSL proxy is both encrypting and decrypting all outgoing web traffic. In most cases, the use of SSL proxy technology also requires manual configuration of the client's browser with a root certificate authority of the appliance server in the trusted list.